{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "LR.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.2"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yYUS-_TGwgog",
        "colab_type": "text"
      },
      "source": [
        "# 第6章 逻辑斯谛回归"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W8eTMpbCwgoh",
        "colab_type": "text"
      },
      "source": [
        "LR是经典的分类方法"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3vYj6djNwgoi",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "回归模型：$f(x) = \\frac{1}{1+e^{-wx}}$\n",
        "\n",
        "其中wx线性函数：$wx =w_0*x_0 + w_1*x_1 + w_2*x_2 +...+w_n*x_n,(x_0=1)$  \n",
        "分类函数: $h(z)=\\frac{1}{1+e^{-z}}=\\frac{1}{1+e^{-w^{T}x}}$  \n",
        "对于任意确定的x和w,有：  \n",
        "$P(y=1|x,w)=h_{w}(x)  $  \n",
        "$P(y=0|x,w)=1−h_{w}(x)  $  \n",
        "服从伯努利分布： \n",
        "$P(y|x,w)=(h_{w}(x))^{y}(1−h_{w}(x))^{1−y}$  \n",
        "似然函数：  \n",
        "$L(w)=∏_{i=1}^{m}P(y^{(i)}|x^{(i)},w)=∏_{i=1}^{m}(h_{w}(x^{(i)}))^{y^{(i)}}(1−h_{w}(x^{(i)}))^{1−y^{(i)}}$  \n",
        "对数似然函数:  \n",
        "$J(w)=logL(w)=∑_{i=1}^{m}(y^{(i)}logh_{w}(x^{(i)})+(1−y^{(i)})log(1−h_{w}(x^{(i)})))$  \n",
        "训练过程就是求解似然函数最大的w：  梯度上升法。  \n",
        "$w_{j}:=w_{j}+α\\frac{∂J(w)}{∂w_{j}} (j=1,2,3...n)$    \n",
        "我们对J(w)求偏导：  \n",
        "$\\frac{∂}{∂w_{j}}J(w)  =∑_{i=1}^{m}(y^{(i)}\\frac{1}{h_{w}(x^{(i)})}\\frac{∂h_{w}(x^{(i)})}{∂w_{j}}+(1−y^{(i)})\\frac{1}{1+h_{w}(x^{(i)})}\\frac{∂h_{w}(x^{(i)})}{∂w_{j}})\\\\=∑_{i=1}^{m}(y^{(i)}\\frac{1}{h_{w}(x^{(i)})}+(1−y^{(i)})\\frac{1}{1+h_{w}(x^{(i)})})\\frac{∂h_{w}(x^{(i)})}{∂w_{j}}\\\\=∑_{i=1}^{m}(y^{(i)}\\frac{1}{h_{w}(x^{(i)})}+(1−y^{(i)})\\frac{1}{1+h_{w}(x^{(i)})})\\frac{1}{(1+e^{−w^{T}x})^{2}}e^{−w^{T}x}\\frac{∂z(x)}{∂w_{j}}\\\\=∑_{i=1}^{m}(y^{(i)}\\frac{1}{h_{w}(x^{(i)})}+(1−y^{(i)})\\frac{1}{1+h_{w}(x^{(i)})})\\frac{1}{(1+e^{−w^{T}x)}}\\frac{e^{−w^{T}x}}{(1+e^{−w^{T}x})}\\frac{∂z(x)}{∂w_{j}}\\\\=∑_{i=1}^{m}(y^{(i)}\\frac{1}{h_{w}(x^{(i)})}+(1−y^{(i)})\\frac{1}{1+h_{w}(x^{(i)})})h_{w}(x^{(i)})(1+h_{w}(x^{(i)}))\\frac{∂w^{T}x}{∂w_{j}}\\\\=∑_{i=1}^{m}(y^{(i)}(1+h_{w}(x^{(i)}))+(1−y^{(i)})h_{w}(x^{(i)}))x^{(i)}_{j}\\\\=∑_{i=1}^{m}(y(i)−h_{w}(x^{(i)}))x^{(i)}_{j}\n",
        "$  \n",
        "\n",
        "梯度上升法迭代公示：\n",
        "$w_{j}:=w_{j}+α∑_{i=1}^{m}(y^{(i)}−h_{w}(x^{(i)}))x^{(i)}_{j}$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RlQPFBq0wgoi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "from sklearn.datasets import load_iris\n",
        "from sklearn.model_selection import train_test_split"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wNcm9Canwgom",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# data\n",
        "def create_data():\n",
        "    iris = load_iris()\n",
        "    df = pd.DataFrame(iris.data, columns=iris.feature_names)\n",
        "    df['label'] = iris.target\n",
        "    df.columns = ['sepal length', 'sepal width', 'petal length', 'petal width', 'label']\n",
        "    data = np.array(df.iloc[:100, [0,1,-1]])\n",
        "    # print(data)\n",
        "    return data[:,:2], data[:,-1]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L1lyUS_iwgoo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X, y = create_data()\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X_cxtXPv2ddJ",
        "colab_type": "code",
        "outputId": "3a22b490-77f1-4f79-b1ca-c012883b16ac",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 107
        }
      },
      "source": [
        "X_train[:5]"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[5.2, 3.5],\n",
              "       [4.7, 3.2],\n",
              "       [6.9, 3.1],\n",
              "       [5.4, 3.4],\n",
              "       [6.3, 3.3]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cjfYWfgG2iY4",
        "colab_type": "code",
        "outputId": "510d457c-bcd0-4eec-e196-7e660ece4ecd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "y[:5]"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0., 0., 0., 0., 0.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fKehxH1crBYT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class LogisticReressionClassifier:\n",
        "    def __init__(self, max_iter=200, learning_rate=0.01):\n",
        "        self.max_iter = max_iter\n",
        "        self.learning_rate = learning_rate\n",
        "        \n",
        "    def sigmoid(self, x):\n",
        "        return 1 / (1 + np.exp(-x))\n",
        "    \n",
        "    def fit(self, X, y):\n",
        "        X = np.hstack((np.ones((X.shape[0],1)),X))\n",
        "        self.weights = np.zeros((X.shape[1], 1))\n",
        "        y = np.expand_dims(y, axis=1)\n",
        "        \n",
        "        for iter_ in range(self.max_iter):\n",
        "            h = self.sigmoid(np.dot(X, self.weights))\n",
        "            error = y - h\n",
        "            self.weights = self.weights + self.learning_rate * np.dot(X.T, error)\n",
        "            \n",
        "    def predict(self, x):\n",
        "        x = np.hstack((np.ones((x.shape[0],1)),x))\n",
        "        pred = self.sigmoid(np.dot(x, self.weights))\n",
        "        if pred > 0:\n",
        "            return 1\n",
        "        else:\n",
        "            return 0\n",
        "    \n",
        "    def score(self, X_test, y_test):\n",
        "        right = 0\n",
        "        X_test = np.hstack((np.ones((X_test.shape[0],1)),X_test))\n",
        "        res = np.dot(X_test, self.weights)\n",
        "        for (result, y) in zip(res, y_test):\n",
        "            if (result > 0 and y == 1) or (result < 0 and y == 0):\n",
        "                right += 1\n",
        "        return right / len(X_test)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0fIAts8nwgot",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lr_clf = LogisticReressionClassifier()\n",
        "lr_clf.fit(X_train, y_train)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pZUAiAlQwgoy",
        "colab_type": "code",
        "outputId": "d1676da1-6fe1-48f6-d42b-c4c306b14b93",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "lr_clf.score(X_test, y_test)"
      ],
      "execution_count": 89,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 89
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eSp_TMsFwgo2",
        "colab_type": "code",
        "outputId": "5dcd88a3-bb03-41ac-e7ee-94b0cea44c2e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 287
        }
      },
      "source": [
        "x_ponits = np.arange(4, 8)\n",
        "y_ = -(lr_clf.weights[1]*x_ponits + lr_clf.weights[0])/lr_clf.weights[2]\n",
        "plt.plot(x_ponits, y_)\n",
        "\n",
        "#lr_clf.show_graph()\n",
        "plt.scatter(X[:50,0],X[:50,1], label='0')\n",
        "plt.scatter(X[50:,0],X[50:,1], label='1')\n",
        "plt.legend()"
      ],
      "execution_count": 90,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f464b26a588>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 90
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8FfW9//HXhySQgBB2CFkISRSQ\nfRFBBALWUhERlyJUW3EDoV5tvbXV/nzYVu+9dbldbG1AxLqLcl0QrUutcNhkaRAEBJcsQBL2hIQl\n+8n398cJijHLJJmcmTP5PB8PH+RMJnM+33Pwk2Hm/f0eMcaglFLKW9o4XYBSSin7aXNXSikP0uau\nlFIepM1dKaU8SJu7Ukp5kDZ3pZTyIG3uSinlQdrclVLKg7S5K6WUB4U79cTdu3c3iYmJTj29UkqF\npK1btx4zxvRoaD/LzV1EwoB0IM8YM73G9+YCjwF51ZueMMYsre94iYmJpKenW316pZRSgIjss7Jf\nY87c7wL2AJ3q+P6rxpg7GnE8pZRSLcTSNXcRiQMuB+o9G1dKKeUOVm+o/hn4JVBVzz7XiMgOEXlN\nROJr20FE5olIuoikHz16tLG1KqWUsqjByzIiMh04YozZKiKpdez2NrDMGFMmIvOB54ApNXcyxiwB\nlgCMHj1a1xpWSjmioqKC3NxcSktLnS6lTpGRkcTFxREREdGkn7dyzX08MENEpgGRQCcRedEYc8OZ\nHYwx+WftvxR4tEnVKKVUEOTm5tKxY0cSExMREafL+Q5jDPn5+eTm5tKvX78mHaPByzLGmPuMMXHG\nmERgNrDq7MYOICIxZz2cQeDGq1JKuVJpaSndunVzZWMHEBG6devWrH9ZNDnnLiIPAunGmJXAnSIy\nA6gECoC5Ta5IKaWCwK2N/Yzm1teo5m6M8QG+6q8fOGv7fcB9zapEKQ9ZsS2Pxz74ggOFJfTpHMU9\nU/szc0Ss02WpVkSXH1DKZiu25XHfGzvJKyzBAHmFJdz3xk5WbMtr8GdV6/H+++/Tv39/UlJSePjh\nh20/vjZ3pWz22AdfUFLh/9a2kgo/j33whUMVKbfx+/389Kc/5b333mP37t0sW7aM3bt32/ocjq0t\no5RXHSgsadR25X52X2bbsmULKSkpJCUlATB79mzeeustzj//fLtK1jN3pezWp3NUo7Yrd2uJy2x5\neXnEx38z1zMuLo68PHsv22lzV8pm90ztT1RE2Le2RUWEcc/U/g5VpJojVC+z6WUZpWx25p/rmpbx\nhpa4zBYbG0tOTs7Xj3Nzc4mNtffvhzZ3pVrAzBGx2sw9ok/nKPJqaeTNucx2wQUX8NVXX5GdnU1s\nbCyvvPIKL7/8cnPK/A69LKOUUvVoicts4eHhPPHEE0ydOpWBAwcya9YsBg0a1NxSv/0cth5NKaU8\npqUus02bNo1p06bZUWKttLkrpVQDQvEym16WUUopD9LmrpRSHqTNXSmlPEibu1JKeZA2d6WU8iBt\n7koRWD9k/MOr6HfvPxj/8Cpdnle1uJtvvpmePXsyePDgFjm+NnfV6un668oJc+fO5f3332+x42tz\nV61eqC4MpYJox3L402D4befAnzuWN/uQEydOpGvXrjYUVzudxKRaPV1/XdVrx3J4+06oqP77UJQT\neAwwdJZzdTVAz9xVq6frr6t6ffTgN439jIqSwHYX0+auWj1df13Vqyi3cdtdQi/LqFZP119X9YqO\nC1yKqW27i2lzV4rQXBhKBcklD3z7mjtARFRgezPMmTMHn8/HsWPHiIuL43e/+x233HJLM4v9hjZ3\npZSqz5mbph89GLgUEx0XaOzNvJm6bNkyG4qrmzZ35Sp2f8q8UrYYOsvVyZjaaHNXrnFmMtGZzPmZ\nyUSANnilGknTMso1dDKRCiZjjNMl1Ku59WlzV66hk4lUsERGRpKfn+/aBm+MIT8/n8jIyCYfQy/L\nKNdoiU+ZV6o2cXFx5ObmcvToUadLqVNkZCRxcU2PW2pzV65xz9T+37rmDjqZSLWMiIgI+vXr53QZ\nLUqbu3INnUyklH0sN3cRCQPSgTxjzPQa32sHPA+MAvKB64wxe22sU7USOplIKXs05sz9LmAP0KmW\n790CHDfGpIjIbOAR4Dob6lPK0zTX3zqcKK2gqsrQuX3boD2npbSMiMQBlwNL69jlSuC56q9fAy4R\nEWl+eUp5l35IiPcdPVnGI+9/zvjfr+KJVRlBfW6rZ+5/Bn4JdKzj+7FADoAxplJEioBuwLFmV6iU\nR9WX69ez99CWU1DMkrVZLE/PodxfxbQhMVw1MrjvaYPNXUSmA0eMMVtFJLU5TyYi84B5AAkJCc05\nlFIhT3P93vPl4ZMs8mWy8tMDtBG4ekQc8yclkdTjnKDXYuXMfTwwQ0SmAZFAJxF50Rhzw1n75AHx\nQK6IhAPRBG6sfosxZgmwBGD06NHunD2gVJBort87tu0/Tpovkw93H6Z92zDmXpTIrRP6ERPt3HvZ\nYHM3xtwH3AdQfeb+ixqNHWAlcCOwEbgWWGXcOvVLKZfQXH9oM8awPuMYaasz2ZiVT3RUBHddci5z\nL0qkS4fg3TitS5Nz7iLyIJBujFkJPA28ICIZQAEw26b6lPIszfWHpqoqwwefHSLNl8nOvCJ6dWrH\n/ZcPZM6YBDq0c8/UIXHqBHv06NEmPT3dkedWSqnGKq+sYsX2PBavySTr6GkSu7Xn9knJXDUylnbh\nYQ0fwCYistUYM7qh/dzza0apFnD/ip0s25yD3xjCRJhzYTz/NXOI02WpEFJS7ueVf+/nqbVZHCgq\nZWBMJ/46ZwTThsQQ1sa9iW9t7sqz7l+xkxc37f/6sd+Yrx9rg1cNKSqu4PmNe3nm470UnC5nTGJX\n/vvqIaSe14NQmMajzV151rLNtXyocfV2be6qLkdOlPL0+mxe2ryfU2WVTBnQk4WpyYxO7Op0aY2i\nzV15lr+O+0l1bVet2/78YhavzeS1rblU+quYPrQPC1KTGRhT24or7qfNXXlWmEitjTwsBP5JrYJn\nz8ETLPJl8s6OA4S3acM1o+KYPzGJxO4dnC6tWbS5K8+ac2H8t665n71dqa37CkhbnclHnx+hQ9sw\nbp2QxC0X96NXp6Z/+pGbaHNXnnXmurqmZdQZxhjWfHmUNF8mW7IL6NI+grsvPY+fjOsb1BUbg0Fz\n7kopz/NXGd7bdZBFvkw+O3CCmOhIbpuQxOwx8bRvG1rnuJpzV65y/VMb2ZBZ8PXj8cldeem2cQ5W\n1LJ0nXZ3KK+s4s1tuSxek0X2sdMkde/Ao9cOZebwWNqGW1rxPGRpc1ctrmZjB9iQWcD1T230ZIM/\ns077mTVjzqzTDmiDD5LTZZUs27KfpeuyOXSilMGxnUi7fiRTB/V29cQjO2lzVy2uZmNvaHuo03Xa\nnVNYXM6zH+/l2Y/3Ulhcwdikrjx67VAmnNs9JCYe2Umbu1I203Xag+9QUSlL12Xx8pb9FJf7+d7A\nXiycnMzIhC5Ol+YYbe5K2UzXaQ+evcdO8+TaTF7fmoffGK4YGsOC1BT6967rQ+NaD23uqsWNT+5a\n6yWY8cmhNZ3bKl2nveV9dqCIRb5M3t15kPCwNsy6II75E5OJ79re6dJcQ5u7anEv3TauVaVldJ32\nlrMlu4A0Xwa+L45yTrtw5k1M5uaLE+nZ0RsTj+ykOXellKsZY1j9xRHSVmeSvu84XTu05ZaL+3HD\n2L5ER0U4XV7Qac5duYrduW+rx9O8eeiq9Ffx7q5DpK3O4PNDJ4ntHMVvrzif6y5IIKpt8D4cI1Rp\nc1ctzu7ct9Xjad48NJVV+nl9ax5Prs1kX34xyT068L8/HMaVw/sQEebtiUd20uauWpzduW+rx9O8\neWg5VVbJy5v3sXRdNkdOljEsLpr7bhjF98/vRZtWMvHITtrcVYuzO/dt9XiaNw8NBafLeXZDNs9t\n3EdRSQXjU7rxx1nDGZ/SrdVNPLKTNnfV4uzOfVs9nubN3e1gUQlPrc1m2Zb9lFT4+f75vVg4OYXh\n8Z2dLs0T9AKWanH3TO1PVMS3b4A1J/dt9Xh2P6+yR9bRU/zytU+Z+Ohqntu4l8uG9ObDn09kyU9G\na2O3kZ65qxZnd+7b6vE0b+4uu/KKSPNl8N6uQ7QNa8OPxiRw28Qk4rroxKOWoDl3pVSLMcawObuA\nv63OYN1Xx+jYLpyfXNSXm8b3o/s57ZwuLyRpzt0jnMppa45cNUdVlWHV50dI82Xwyf5Cup/Tll/+\noD83jO1Lp8jWN/HICdrcXcypnLbmyFVTVfqreGdH4BOPvjh8krguUTx05SB+ODqeyAideBRMekPV\nxerLabvheZ2qT7lPaYWfFzbtY/IffPzs1e0YDH+6bhirf5HKj8clamN3gJ65u5hTOW3NkSurTpZW\n8OKm/Ty9Pptjp8oYHt+ZB6YP4pIBPXXikcO0ubuYUzltzZGrhuSfKuOZDXt5buNeTpZWMuHc7ixI\nHc64JJ145BZ6WcbFnMppa45c1SWvsITfrvyM8Y+s4m++DC5O6c7KO8bzwi0XclFy6/soOzfTM3cX\ncyqnrTlyVVPGkZMs8mXx1vY8AK4aEcv8Scmk9DzH4cpUXRrMuYtIJLAWaEfgl8Frxpjf1NhnLvAY\nkFe96QljzNL6jqs5d6Xc79OcQtJ8Gfxz92HahbdhzpgEbpuQpJfeHGRnzr0MmGKMOSUiEcB6EXnP\nGLOpxn6vGmPuaEqxKnTdv2Inyzbn4DeGMBHmXBjPf80c0uT93J7rbw2MMWzMzCfNl8n6jGN0igzn\njskpzL0okW468ShkNNjcTeDU/lT1w4jq/5yZ1qpc5f4VO3lx0/6vH/uN+frx2Y3b6n5uz/V7XVWV\n4cM9h0nzZfJpTiE9OrbjvssG8KMLE+ioE49CjqUbqiISJiLbgSPAh8aYzbXsdo2I7BCR10Qk3tYq\nlSst25xjabvV/dye6/eqCn8Vr2/NZeqf1zL/ha0cP13Of181mHW/nMz8Scna2EOUpRuqxhg/MFxE\nOgNvishgY8yus3Z5G1hmjCkTkfnAc8CUmscRkXnAPICEhIRmF6+c5a/jfk3N7Vb3c3uu32tKK/ws\nT8/hyTVZ5BWWMKB3Rx6fPZzLh8QQrp94FPIalZYxxhSKyGrgB8Cus7bnn7XbUuDROn5+CbAEAjdU\nG12tcpUwkVobd1iNOJzV/dye6/eKE6UVvLBxH89syObYqXJG9e3Cg1cOYsqAnhpl9JAGfz2LSI/q\nM3ZEJAq4FPi8xj4xZz2cAeyxs0jlTnMurP3qW83tVvdze64/1B09WcYj73/O+N+v4rEPvmBQn2he\nnTeW124fxyUDe2lj9xgrZ+4xwHMiEkbgl8FyY8w7IvIgkG6MWQncKSIzgEqgAJjbUgUr9zhzM7Sh\nFIzV/dye6w9VOQXFLFmbxfL0HMr9VUwbEsOCSckMjo12ujTVgnQ9d6U86svDJ1nky2TlpwdoI3D1\niDjmT0oiqYdOPAplup67R9idv7aaN7f7eE6tD2/3eF1vx3K2vf8MaYVj+bBqNFFhhrkXJXHrhH7E\nRHvzHoKqnTZ3F7M7f201b2738ZxaH97u8bqZMYb1/3ydtLX72ei/g2hOcVfY68yNWkOXhN9D9PlO\nl6iCTPNOLmZ3/tpq3tzu4zm1Przd43WjqirDezsPMuOJDfx4dRRZ/l7cH/4iH7f7D34e8TpdKo/B\nRw86XaZygJ65u5jd+WureXO7j+fU+vB2j9dNyiurWLE9j8VrMsk6eprEbu15OPwprgpbRzup/PbO\nRbnOFKkcpc3dxezOX1vNm9t9PKfWh7d7vG5QUu7nlX/v56m1WRwoKmVgTCf+OmcE04bEEPb4f0BR\n5Xd/KDou+IUqx+llGRezO39tNW9u9/GcWh/e7vE6qai4gr9+9BXjH1nF797eTVyX9jxz0wW8e+fF\nXDGsD2FtBC55ACJq/CKMiApsV62Onrm7mN35a6t5c7uP59T68HaP1wlHTpTy9PpsXty0j9PlfqYM\n6MnC1GRGJ3b97s5DZwX+/OjBwKWY6LhAYz+zXbUqmnNXyoX25xezeG0mr23NpdJfxfShfViQmszA\nmE5Ol6Ycpjl3FRRO5de9as/BEyzyZfLOjgOEt2nDNaPimD8xicTuHZwurX47luu/GFxGm7tqMqfy\n6160dV8Baasz+ejzI3RoG8atE5K45eJ+9OoU6XRpDduxHN6+Eyqqb4YX5QQegzZ4B2lzV01WXy79\n7KZtdb/WxhjDmi+PkubLZEt2AV3aR3D3pedx47hEotuH0BrqHz34TWM/o6IksF2bu2O0uasmcyq/\nHur8VYb3dh1kkS+Tzw6cICY6kgemn8/sMfG0bxuC/0vWlaPXfL2jQvBvknILp/Lroaqs0s+bn+Tx\n5Nosso+dJql7Bx69digzh8fSNjyEU8nRcYFLMbVtV44J4b9RymlO5ddDzemySpauy2LSoz7ufWMn\nHdqFkXb9SD68exKzRseHdmMHzde7lJ65qyZzKr8eKgqLy3n24708+/FeCosrGJvUlUevHcqEc7t7\n64MxNF/vSppzV8pmh4pKWboui5e37Ke43M/3BvZi4eRkRiZ0cbo05QGac3eI3Xluq8dzat1yza9/\nI/vYaZ5ck8kbn+ThN4YZw/pw+6Rk+vfu2PyDeylH7qWxWOHQeLW528juPLfV4zm1brnm1wM+O1DE\nIl8m7+48SHhYG667IJ55E5OI79renifwUo7cS2OxwsHxhvidHHexez1yq8dzat1yu8cbarZkFzD3\nmS1c/pf1+L44yryJyaz/1WQemjnYvsYO9efIQ42XxmKFg+PVM3cb2Z3ntno8p9Ytb435dWMMq784\nQtrqTNL3Hadbh7bcM7U/N4ztS3RUC0088lKO3EtjscLB8Wpzt5HdeW6rx3Nq3fLWlF+v9Ffxj52B\niUefHzpJbOcofjdjELNGxxPVNqzhAzSHl3LkXhqLFQ6OVy/L2MjuPLfV4zm1bnlryK+XVvh5afM+\npvxhDXe9sp3KKsP//nAYvntSufGixJZv7OCtHLmXxmKFg+PVM3cb2Z3ntno8p9Yt93J+/VRZJS9v\n3sfSddkcOVnGsLhofj1tFN8/vxdt2gQ5o+6lHLmXxmKFg+PVnLtSZyk4Xc6zG7J5buM+ikoqGJ/S\njYWpKVyU3M1bE49UyNKcu0e0tty8Uw4UlvDUuixe2ZJDSYWfqYN6sTA1hWHxnZ0uTdnlnbth67Ng\n/CBhMGouTP9j04/n8ry+NncXa225eSdkHj3FYl8mK7bnYQxcOTyWBalJpPS0YeKRco937ob0p795\nbPzfPG5Kgw+BvL7eUHWx1pabD6aduUUsfGkr3/vjGlZ+eoAfjUnAd08qf5g1TBu7F219tnHbGxIC\neX09c3ex1pabb2nGGDZlFZDmy2DdV8foGBnOwtRkbhrfj+7ntHO6PNWSjL9x2xsSAnl9be4u1tpy\n8y2lqsrw0edHSPNlsG1/Id3PacevfjCA68cm0CkyhD7xSDWdhNXeyKWJUdYQyOvrZRkXa225ebtV\n+qt4c1suP3h8Lbc9n87Rk2U8NHMw6381mQWpydrYW5NRcxu3vSEhkNfXM3cXa225ebuUVvj5v/Qc\nnlybRe7xEvr36sifrxvO9KExhIfp+UyrdOamqV1pmRDI6zeYcxeRSGAt0I7AL4PXjDG/qbFPO+B5\nYBSQD1xnjNlb33E1567sdrK0ghc37efp9dkcO1XGiITOLExN4ZIBPYM/8UipFmJnzr0MmGKMOSUi\nEcB6EXnPGLPprH1uAY4bY1JEZDbwCHBdkyp3Kav5cLevb241vx5K4z12qoxnNmTz/MZ9nCytZMK5\n3VmYOoKxSV2Rnf8Hj9t4dmU1K213Btrtx3OS1bF4acwWNNjcTeDU/lT1w4jq/2qe7l8J/Lb669eA\nJ0REjFPTX21mNR/u9vXNrebXQ2W8uceLeWptFq+m51BWWcVlg3uzYFIKQ+KiAzvYnUW2mpW2+3nd\nfjwnWR2Ll8ZskaULkCISJiLbgSPAh8aYzTV2iQVyAIwxlUAR0M3OQp1kNR/u9vXNrebX3T7erw6f\n5O7l20l9zMdLm/dzxdA+/OvuSaRdP+qbxg72Z5GtZqXtfl63H89JVsfipTFbZOmGqjHGDwwXkc7A\nmyIy2Bizq7FPJiLzgHkACQkJjf1xx1jNh7t9fXOr+XW3jnd7TiFpqzP45+7DREWE8eNxfbltQlLd\n0VC7s8hWs9J2P6/bj+ckq2Px0pgtalR0wBhTCKwGflDjW3lAPICIhAPRBG6s1vz5JcaY0caY0T16\n9GhaxQ6oq3nU3G51P6fUlVOvud1N4zXGsCHjGNcv3cTMv21gU1Y+d05JYcO9U/jNFYPqf666MsdN\nzSLXlYmuud3u53X78ZxkdSxeGrNFDTZ3EelRfcaOiEQBlwKf19htJXBj9dfXAqu8cr0drOfD3b6+\nudX8uhvGW1VleH/XIWb+bQPXL93MV4dP8etpA/j4vku4+/v96dqhbcMHsTuLbDUrbffzuv14TrI6\nFi+N2SIrl2VigOdEJIzAL4Plxph3RORBIN0YsxJ4GnhBRDKAAmB2i1XsAKv5cLevb241v+7keCv8\nVby1/QCL12SSceQUfbu153+uGsLVI2OJjGjkbEK7s8hWs9J2P6/bj+ckq2Px0pgt0vXclSuUlPtZ\nnp7DkrVZ5BWWMKB3RxZOTmHa4N468Uips+h67g5xQ+47lBSVVPDipn38fX02+afLGd23Cw/NHMTk\n/j3d+eEYbs9U211fS4xDM/tBoc3dRk7nvkPJkZOl/H39Xl7ctI9TZZWk9u/BwtQUxvTr6nRpdXN7\nptru+lpiHJrZDxq9LGOj8Q+vqnXVxdjOUWy4d4oDFblPTkExT67NZHl6LpX+KqYNiWFBajKD+kQ3\n/MNO+9PgOlYCjIef72r8fm6vryXGYfcxnXqtHaSXZRzg9py7k744dJJFvgze3nGQMBGuGRXLvInJ\n9OvewenSrHN7ptru+lpiHJrZDxpt7jaye/11L/hk/3HSVmfyrz2Had82jJvHJ3LLxUn0jo50urTG\ns7qGt1NrfdtdX0uMw+5jhsC66k7RGIKN3J5zDxZjDGu/PMrsJRu5Ou1j0vcV8LPvncuGX03h/11+\nfmg2dnB/ptru+lpiHJrZDxo9c7eR23PuLc1fZfjgs0Ms8mWyM6+I3p0iuf/ygcwZk0CHdh74q+b2\nTLXd9bXEODSzHzR6Q1U1W3llFSu25bF4bSZZR0/Tr3sHbp+UxMwRsbQLb+LHmCmlaqU3VFWLKy6v\n5JUtOTy1LouDRaWcH9OJJ340gssGxxDW2j8cw+q6705xe33g/rkCLqfNXTVaUXEFz23cyzMbsjle\nXMGYfl35/dVDmHReD3dOPAo2q+u+O8Xt9YH75wqEAL0soyw7cqKUpeuzeWnTPk6X+7lkQE8WTk5m\nVF8XTzxywu+61r48sITBbwqCX09Nbq8P3D9XwEF6WUbZZl/+aRavyeL1rblUVlUxfWgfFqQmMzCm\nk9OluZPVdd+d4vb6wP1zBUKANndVpz0HT5Dmy+QfOw4Q3qYN146OY/7EJPp2C6GJR06QsLrPjN3A\n7fWB++cKhADNuavvSN9bwM3P/pvLHl/Hqj2HuW1CEut/NZn/uWqINnYrrK777hS31wfunysQAvTM\nXQGBiUe+L4+yaHUmW/YW0LVDW/7z0vP4ybhEottHOF1eaLG67rtT3F4fuH+uQAjQG6qtnL/K8O7O\ngyzyZbL74An6REdy28QkrrsgnvZt9Xe/Um6jN1RVvcoq/bzxSR5Prslkb34xST068Ni1Q7lyeCxt\nwx28Wuf2jLHd9dmdN3f766eCRpt7K3O6rJJlW/bz1LosDp8oY0hsNIuuH8n3B/V2fuKR2zPGdtdn\nd97c7a+fCiq9LNNKHD9dzrMf7+W5jXspLK5gXFI3Fk5O5uKU7u6ZeOT2jLHd9dmdN3f766dsoZdl\nFACHikp5al0Wy7bsp7jcz6Xn92JBajIjE7o4Xdp3uT1jbHd9dufN3f76qaDS5u5R2cdOs9iXyRvb\ncqkyMGNYH26flEz/3h2dLq1ubs8Y212f3Xlzt79+Kqg05+4xu/KK+OnLnzDlDz7e3J7H7AsS8P0i\nlT9dN9zdjR3cnzG2uz678+Zuf/1UUOmZuwcYY9iSXUCaL5M1Xx6lY7twbp+UzM3j+9GjYzuny7PO\n7Rlju+uzO2/u9tdPBZXeUA1hxhhWfX6ENF8mW/cdp1uHttx8cT9uGNuX6CideKSUF+kNVQ+r9Ffx\nj+qJR58fOkls5ygevHIQs0bHExnhovVBQond+XCrx9Ncumoh2txDSGmFn9c/yeXJNVnsLygmpec5\n/OGHw5gxvA8RYXr7pMnszodbPZ7m0lUL0ssyIeBUWSUvbdrH0vXZHD1ZxrD4zixMTebSgb1o4/TE\nIy+wOx9u9XiaS1dNoJdlPCD/VFlg4tHHezlRWsnFKd15/LrhjEvu5p6JR15gdz7c6vE0l65akDZ3\nFzpQWMKStVm88u/9lFZUMXVQLxampjAsvrPTpXmT3flwq8fTXLpqQdrcXSTjyCkWr8lkxbY8AK4c\nHsuC1CRSero8nx7qLnng29e+oXn5cKvHs/t5lTqLNncX2JlbRJovg/c/O0S78DbcMLYvt07oR1yX\n9k6X1jrYnQ+3ejzNpasW1OANVRGJB54HegEGWGKMebzGPqnAW0B29aY3jDEP1nfc1n5D1RjDxqx8\nFvkyWffVMTpGhnPjuETmjk+k+zkhNPFIKRVUdt5QrQT+0xjziYh0BLaKyIfGmN019ltnjJnelGJb\nk6oqw7/2HCbNl8n2nEK6n9OOX/1gADeMTaBjpIcnHrk9z6259ObR18V1GmzuxpiDwMHqr0+KyB4g\nFqjZ3FU9Kv1VvL3jAIt8mXx5+BRxXaJ4aOZgfjgqzvsTj9ye59ZcevPo6+JKjZr5IiKJwAhgcy3f\nHicin4rIeyIyyIbaPKG0ws8LG/eS+r8+fv7qpwjCn68bju8Xqfx4bF/vN3YInNGdfdMQAo8/qvfK\nXfBYrc/t43CKvi6uZPmGqoicA7wO/MwYc6LGtz8B+hpjTonINGAFcG4tx5gHzANISEhoctGh4ERp\nBS9u2sff12dz7FQ5IxM689usXfZ5AAAK4ElEQVQrBjFlQM/WN/HI7XluzaU3j74urmSpuYtIBIHG\n/pIx5o2a3z+72Rtj3hWRNBHpbow5VmO/JcASCNxQbVblLnXsVBl/X5/NCxv3cbKskonn9WBhajIX\n9uvaeiceuT3Prbn05tHXxZUavCwjgY70NLDHGFPrWqQi0rt6P0RkTPVx8+0s1O1yjxfzwFu7GP/w\nKhatyWTCed15+46Lef7mMYxNauUzSt2+zrjV+tw+Dqfo6+JKVs7cxwM/BnaKyPbqbb8GEgCMMYuB\na4EFIlIJlACzjVOL1gTZV4dPsmhNJiu3H0AErhoRy/xJyST3OMfp0tzD7XluzaU3j74urqQLhzXR\n9pxC0lZn8M/dh4mKCGPOmARundCPPp2jGv5hpZRqIl04rAUYY9iQkU+aL4OPM/PpFBnOnVNSmDu+\nH107tHW6PHu1ttzyO3fb94lISrmANncLqqoM/9x9mEW+DD7NLaJnx3b8etoAfnRhX85p58GXsLXl\nlt+5G9Kf/uax8X/zWBu8ClEe7Ez2qfBX8db2Ayxek0nGkVP07dae/7lqCFePjPV2Pr2+3LIXm/vW\nZ+vers1dhSht7rUoKffz6r/389S6bPIKSxjQuyN/mTOCaYN7E94aPvGoteWWjb9x25UKAdrcz1JU\nUsELG/fyzIa95J8uZ3TfLjw0cxCT+/dsXVHG1pZblrDaG7l4+F9nyvO0uQNHTpby9PpsXtq0n1Nl\nlaT278HC1BTG9OvqdGnOaG3rjI+a++1r7mdvVypEtermnlNQzJNrM1menkulv4ppQ2JYkJrMoD7R\nTpfmrNaWWz5zXV3TMspDWmXO/YtDJ1nky+DtHQcJE+GaUbHMn5hMYvcOjtSjlFJWac69Flv3HWeR\nL4N/7TlC+7Zh3Dw+kVsuTqJ3dGTQa1mxLY/HPviCA4Ul9OkcxT1T+zNzRGzQ62g2r+ThvTIOp+jr\n5zqeb+7GGNZ9dYy/rc5gc3YBndtH8LPvncuN4xLp4tDEoxXb8rjvjZ2UVARu4uUVlnDfGzsBQqvB\neyUP75VxOEVfP1fy7GUZf5Xhg88OkebLYFfeCXp3iuTWCf2YMyaBDg5PPBr/8CryCku+sz22cxQb\n7p3iQEVN9KfBdaRq4uHnu4JfT1N5ZRxO0dcvqFrtZZnyyipWbMtj8ZpMso6dpl/3DjxyzRBmjoil\nXbg7om0Hamns9W13La/k4b0yDqfo6+dKnmnuxeWVLNuSw9J1WRwsKuX8mE488aMRXDY4hjCXfThG\nn85RtZ65h9yiY17Jw3tlHE7R18+VQn66ZWFxOY//6yvGP7yKh97ZTXzX9jx70wX8486LmT60j+sa\nO8A9U/sTVWP5gqiIMO6Z2t+hiprIK+t4e2UcTtHXz5VC9sz98IlSlq7L4uXN+zld7ueSAT1ZODmZ\nUX3dP/HozE3TkE/LeCUP75VxOEVfP1cKuRuqOQXFpPkyeX1rLpVVVUwf2ocFqckMjOnUAlUqpZS7\nePaG6p6DJ3h9ay7Xjo5j/sQk+nbTiUdKfc3uvLnm10NWyDX37w3sxfp7J9OzY/AnHinlanbnzTW/\nHtJC7oZqmzaijV2p2tS3Dr8bjqeCKuSau1KqDnbnzTW/HtK0uSvlFXXlypuaN7f7eCqotLkr5RV2\n5801vx7StLkr5RVDZ8EVfwms6YIE/rziL02/+Wn38VRQhVzOXSmlWjOrOXc9c1dKKQ/S5q6UUh6k\nzV0ppTxIm7tSSnmQNnellPIgbe5KKeVB2tyVUsqDtLkrpZQHNdjcRSReRFaLyG4R+UxE7qplHxGR\nv4hIhojsEJGRLVOucp0dy+FPg+G3nQN/7ljudEVKKayt514J/Kcx5hMR6QhsFZEPjTG7z9rnMuDc\n6v8uBBZV/6m8TNf7Vsq1GjxzN8YcNMZ8Uv31SWAPUPPDPq8EnjcBm4DOIhJje7XKXXS9b6Vcq1HX\n3EUkERgBbK7xrVgg56zHuXz3FwAiMk9E0kUk/ejRo42rVLmPrvetlGtZbu4icg7wOvAzY8yJpjyZ\nMWaJMWa0MWZ0jx49mnII5Sa63rdSrmWpuYtIBIHG/pIx5o1adskD4s96HFe9TXmZrvetlGtZScsI\n8DSwxxjzxzp2Wwn8pDo1MxYoMsYctLFO5Ua63rdSrmUlLTMe+DGwU0S2V2/7NZAAYIxZDLwLTAMy\ngGLgJvtLVa40dJY2c6VcqMHmboxZD0gD+xjgp3YVpZRSqnl0hqpSSnmQNnellPIgbe5KKeVB2tyV\nUsqDtLkrpZQHaXNXSikP0uaulFIeJIGIugNPLHIU2NfEH+8OHLOxHCd5ZSw6DnfxyjjAO2Oxaxx9\njTENLs7lWHNvDhFJN8aMdroOO3hlLDoOd/HKOMA7Ywn2OPSyjFJKeZA2d6WU8qBQbe5LnC7ARl4Z\ni47DXbwyDvDOWII6jpC85q6UUqp+oXrmrpRSqh6ub+4iEiYi20TknVq+105EXhWRDBHZXP0Zr67U\nwDjmishREdle/d+tTtRohYjsFZGd1XWm1/J9EZG/VL8nO0RkpBN1NsTCOFJFpOis98SVHy8lIp1F\n5DUR+VxE9ojIuBrfD5X3o6FxhMr70f+sGreLyAkR+VmNfYLynlj5sA6n3QXsATrV8r1bgOPGmBQR\nmQ08AlwXzOIaob5xALxqjLkjiPU0x2RjTF153cuAc6v/uxBYVP2nG9U3DoB1xpjpQaumaR4H3jfG\nXCsibYH2Nb4fKu9HQ+OAEHg/jDFfAMMhcEJH4ONG36yxW1DeE1efuYtIHHA5sLSOXa4Enqv++jXg\nkuqPBXQVC+PwkiuB503AJqCziMQ4XZQXiUg0MJHAx2BijCk3xhTW2M3174fFcYSiS4BMY0zNyZpB\neU9c3dyBPwO/BKrq+H4skANgjKkEioBuwSmtURoaB8A11f9Ee01E4uvZz2kG+KeIbBWRebV8/+v3\npFpu9Ta3aWgcAONE5FMReU9EBgWzOIv6AUeBZ6ov+S0VkQ419gmF98PKOMD970dNs4FltWwPynvi\n2uYuItOBI8aYrU7X0hwWx/E2kGiMGQp8yDf/GnGji40xIwn80/KnIjLR6YKaqKFxfEJgmvcw4K/A\nimAXaEE4MBJYZIwZAZwG7nW2pCaxMo5QeD++Vn1paQbwf07V4NrmTuCDuWeIyF7gFWCKiLxYY588\nIB5ARMKBaCA/mEVa0OA4jDH5xpiy6odLgVHBLdE6Y0xe9Z9HCFxLHFNjl6/fk2px1dtcpaFxGGNO\nGGNOVX/9LhAhIt2DXmj9coFcY8zm6sevEWiSZwuF96PBcYTI+3G2y4BPjDGHa/leUN4T1zZ3Y8x9\nxpg4Y0wigX/erDLG3FBjt5XAjdVfX1u9j6uC+1bGUeN62wwCN15dR0Q6iEjHM18D3wd21dhtJfCT\n6kTAWKDIGHMwyKXWy8o4RKT3mfs3IjKGwP8rrjpxMMYcAnJEpH/1pkuA3TV2c/37YWUcofB+1DCH\n2i/JQJDek1BIy3yLiDwIpBtjVhK4AfOCiGQABQSaZ0ioMY47RWQGUElgHHOdrK0evYA3q/8fCwde\nNsa8LyK3AxhjFgPvAtOADKAYuMmhWutjZRzXAgtEpBIoAWa77cSh2n8AL1VfBsgCbgrB9wMaHkeo\nvB9nThguBeaftS3o74nOUFVKKQ9y7WUZpZRSTafNXSmlPEibu1JKeZA2d6WU8iBt7kop5UHa3JVS\nyoO0uSullAdpc1dKKQ/6/ywsNWlpqfDcAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "33BTHkrY2sDj",
        "colab_type": "text"
      },
      "source": [
        "##### 作业"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f2n8BPac2qT8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "My_X = np.array([[3,3,3],\n",
        "              [4,3,2],\n",
        "              [2,1,2],\n",
        "              [1,1,1],\n",
        "              [-1,0,1],\n",
        "              [2,-2,1]])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hI2xETv73BaD",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "7483cda4-0715-4e51-ab18-3776573d4b47"
      },
      "source": [
        "My_X.shape"
      ],
      "execution_count": 92,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(6, 3)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 92
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jG0OK50n3DNj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "My_y = np.array([1,1,1,0,0,0])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hZvU2HTM3Kmb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "My_clf = LogisticReressionClassifier()\n",
        "My_clf.fit(My_X, My_y)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7sMdxyTw3RKM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "c2104b89-60fd-489b-9622-c37c5a6dd768"
      },
      "source": [
        "test_point = np.array([1,2,-2])\n",
        "My_clf.predict(np.expand_dims(test_point, axis=0))"
      ],
      "execution_count": 110,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 110
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "eyezi6wCwgo5",
        "colab_type": "text"
      },
      "source": [
        "## sklearn\n",
        "\n",
        "### sklearn.linear_model.LogisticRegression\n",
        "\n",
        "solver参数决定了我们对逻辑回归损失函数的优化方法，有四种算法可以选择，分别是：\n",
        "- a) liblinear：使用了开源的liblinear库实现，内部使用了坐标轴下降法来迭代优化损失函数。\n",
        "- b) lbfgs：拟牛顿法的一种，利用损失函数二阶导数矩阵即海森矩阵来迭代优化损失函数。\n",
        "- c) newton-cg：也是牛顿法家族的一种，利用损失函数二阶导数矩阵即海森矩阵来迭代优化损失函数。\n",
        "- d) sag：即随机平均梯度下降，是梯度下降法的变种，和普通梯度下降法的区别是每次迭代仅仅用一部分的样本来计算梯度，适合于样本数据多的时候。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fucfcIUowgo5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.linear_model import LogisticRegression"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ES6AI5wdwgo8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "clf = LogisticRegression(max_iter=200)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F1XhyxI2wgo-",
        "colab_type": "code",
        "outputId": "ff8efb48-026d-4c07-ffff-66d73e518630",
        "colab": {}
      },
      "source": [
        "clf.fit(X_train, y_train)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "          intercept_scaling=1, max_iter=200, multi_class='ovr', n_jobs=1,\n",
              "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
              "          verbose=0, warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5PDQagu0wgpB",
        "colab_type": "code",
        "outputId": "51f58cce-b098-4038-a1cc-fb753132db49",
        "colab": {}
      },
      "source": [
        "clf.score(X_test, y_test)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9666666666666667"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5eWQ-dMgwgpE",
        "colab_type": "code",
        "outputId": "caade4c0-a088-4dde-a75b-9a8a1c63b1eb",
        "colab": {}
      },
      "source": [
        "print(clf.coef_, clf.intercept_)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 1.94474283 -3.29077674]] [-0.53064339]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rOP29ZLMwgpH",
        "colab_type": "code",
        "outputId": "24c023fd-5afb-4cde-fb24-3e5c505484bd",
        "colab": {}
      },
      "source": [
        "x_ponits = np.arange(4, 8)\n",
        "y_ = -(clf.coef_[0][0]*x_ponits + clf.intercept_)/clf.coef_[0][1]\n",
        "plt.plot(x_ponits, y_)\n",
        "\n",
        "plt.plot(X[:50, 0], X[:50, 1], 'bo', color='blue', label='0')\n",
        "plt.plot(X[50:, 0], X[50:, 1], 'bo', color='orange', label='1')\n",
        "plt.xlabel('sepal length')\n",
        "plt.ylabel('sepal width')\n",
        "plt.legend()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x1499696b710>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VPW5+PHPQwLEyKIsKhCSQOIGKKu4IDtVKxatWpeit26lArb2drOtXe7PXm+rt4tt2cSltYrora0bVdsCCQgICAKioJAQlgCyBAhLDCHJ8/vjTEKMk+QkOWfmzMnzfr3mNZkzZ8483xnIk3O+3+f7FVXFGGOMAWgV7wCMMcYEhyUFY4wx1SwpGGOMqWZJwRhjTDVLCsYYY6pZUjDGGFPNkoIxxphqvicFEUkSkTUiMi/Kc3eIyD4RWRu53eN3PMYYY+qWHIP3uB/YCHSo4/kXVfW+GMRhjDGmAb4mBRFJA8YDDwPf8eKYXbp00czMTC8OZYwxLcbq1av3q2rXhvbz+0zhMeAHQPt69rlBREYAm4D/VNUd9R0wMzOTVatWeRiiMcaEn4hsc7Ofb30KInINsFdVV9ez2+tApqpeCMwHnqnjWJNEZJWIrNq3b58P0RpjjAF/O5qHARNEZCvwAjBGRJ6ruYOqFqnq8cjDJ4DB0Q6kqrNVdYiqDunatcGzH2OMMU3kW1JQ1R+papqqZgK3AAtV9baa+4hItxoPJ+B0SBtjjImTWIw++gwReQhYpaqvAd8SkQlAOXAAuCPW8RhjjFsnTpygsLCQ0tLSeIdSp5SUFNLS0mjdunWTXi+Jtp7CkCFD1DqajTHxUFBQQPv27encuTMiEu9wPkdVKSoq4siRI/Tq1eszz4nIalUd0tAxrKLZmACaMwcyM6FVK+d+zpx4R2QASktLA5sQAESEzp07N+tMJuaXj4wx9ZszByZNgpIS5/G2bc5jgIkT4xeXcQQ1IVRpbnx2pmBMwDz44MmEUKWkxNlujN8sKRgTMNu3N267aVneeustzj33XLKzs/nVr37l+fEtKRgTMOnpjdtugsvrvqGKigqmTp3Km2++yYYNG5g7dy4bNmzwItRqlhSMCZiHH4bU1M9uS011tpvEUdU3tG0bqJ7sG2pOYli5ciXZ2dn07t2bNm3acMstt/Dqq696FzSWFIwJnIkTYfZsyMgAEed+9mzrZE40fvQN7dy5k549e1Y/TktLY+fOnU0/YBQ2+siYAJo40ZJAovOjbyhaXZnXo6HsTMEYY3zgR99QWloaO3acnEi6sLCQ7t27N/2AUVhSMMYYH/jRN3TRRRexefNmCgoKKCsr44UXXmDChAnNC7QWSwrGGOMDP/qGkpOTmTZtGldeeSXnn38+N910E3379vUuaKxPwRhjfONH39DVV1/N1Vdf7e1Ba7AzBWOMMdUsKRhjjKlmScEYY0w1SwrGGGOqWVIwxhhTzZKCMR6wRXFMWFhSMKaZ/Jj4zJi63HXXXZxxxhn069fPl+NbUjCmmWxRHFOngjnwSiY838q5L2j+Xwp33HEHb731VrOPUxdLCsY0ky2KY6IqmAMrJ0HJNkCd+5WTmp0YRowYQadOnbyJMQpLCsY0ky2KY6Ja9yBU1DqFrChxtgeYJQVjmskWxTFRldRxqljX9oCwpGBMM9miOCaq1DpOFevaHhCWFIzxwMSJsHUrVFY695YQDP0fhqRap5BJqc72ALOkYELDagVMoPSaCENnQ2oGIM790NnO9ma49dZbufTSS/n4449JS0vjqaee8ibeCJs624RCVa1A1dDQqloBsL/aTRz1mtjsJFDb3LlzPT1ebXamYELBagWM8YYlBRMKVitgjDcsKZhQsFoBEyuqGu8Q6tXc+CwpmFCwWgETCykpKRQVFQU2MagqRUVFpKSkNPkY1tFsQqGqM/nBB51LRunpTkKwTmbjpbS0NAoLC9m3b1+8Q6lTSkoKaWlpTX69BDXj1WXIkCG6atWqeIdhjDEJRURWq+qQhvbz/fKRiCSJyBoRmRflubYi8qKI5InIChHJ9DseY8LEajOM12LRp3A/sLGO5+4GDqpqNvA74JEYxGNMKNg6DsYPviYFEUkDxgNP1rHLtcAzkZ9fAsaKiPgZkzFhYbUZxg9+nyk8BvwAqKzj+R7ADgBVLQeKgc61dxKRSSKySkRWBbmDx5hYstoM4wffkoKIXAPsVdXV9e0WZdvner5VdbaqDlHVIV27dvUsRmMSmdVmtByHSsp4bP4mluXv9/29/DxTGAZMEJGtwAvAGBF5rtY+hUBPABFJBjoCB3yMyZjQsNqM8Nt7pJRfvrGRYb9ayGPzN7M8v8j39/StTkFVfwT8CEBERgHfU9Xbau32GvA14B3gRmChJtoYWWPixGozwqvwYAmzF2/hxXd3cKKikvEXdmfKqCzO79bB9/eOefGaiDwErFLV14CngGdFJA/nDOGWWMdjTCKbONGSQJhs2XeUGbn5vLJmJwDXD+rB5FHZ9OpyasxiiElSUNVcIDfy889qbC8FvhKLGIxprClTnBXUKiogKckZ7jljRryjMmG0Yddhpufm8cb63bRJasVtl2Tw9RG96XHaKTGPxaa5MCaKKVNg5syTjysqTj62xGC88t72g0xfmMeCj/bSrm0y3xiRxd2X96Jr+7Zxi8mmuTAmiuRkJxHUlpQE5eWxj8eEh6qyLL+IaQvzeGdLEaeltuauYb342qWZdExt7dv7up3mws4UjIkiWkKob7sxDVFVFmzcy7ScPNbuOETX9m158Orz+erF6ZzaNji/ioMTiTEBkpRU95mCMY1RUam8sX4303Py+OiTI/Q47RR+cV0/vjI4jZTWwfsHZUnBmCgmTfpsn0LN7ca4UVZeyStrdjJzUT4F+4+R1fVUfvOV/kwY0J3WScFdysaSgjFRVHUm2+gj01ilJyp48d0dPL4on13FpfTt3oEZEwdxZd+zSGoV/KndrKPZGGM8cKT0BHNWbOfJtwvYf/Q4QzJOZ+qYbEad05UgzPNpHc3GGBMDB4+V8adlW/nz0gIOl5Yz/OwuTB09kIt7dQpEMmis4F7YMiZi3DgQOXkbNy7eEfnPFs8Jvr2HS/mfNzYy7JGF/GHBZi7u3ZlXpw7j2bsv5pLenRMyIYCdKZiAGzcOFiz47LYFC5zt8+fHJya/VS2eU7VWQtXiOWBTWgTBjgOReYlW7aC8opIv9e/O5FFZnHeW//MSxYL1KZhAq++PrQT7p+taZqaTCGrLyICtW2MdjamSv+8oM3LyeXXtTkTghkFp3Dsyi8wYzkvUHNanYEyCssVzguXDXcXMyMnnjQ920zbZmZdo0ojedI/DvESxYEnBmIBJT49+pmCL58TW6m0HmLYwj5yP99G+bTKTR2Zx1+W96NIufvMSxYIlBRNoY8d+vk+hantYPfzwZ/sUwBbPiRVVZWleEdNyNrN8ywFOT23Nd79wDv9xWSYdT/FvXqIgsaRgAm3+/M93No8dG95OZrDFc+KhslKZv3EP03PzWbfjEGd2aMtPxp/PrUODNS9RLFhHszGmxaqoVOa9v4sZOfl8vOcIPTudwr0js7hxcBptk4M3L1FzuO1otjoFE3hej9l3ezyrFQivsvJKXnx3O2N/k8v9L6ylQpXf3tSfnO+OYuLFGaFLCI3Rss6LTMLxesy+2+NZrUA4fVpWwQvvbmf24i3sLi6lX48OzLptEFf0OYtWCTAvUSzY5SMTaF6P2Xd7PKsVCJcjpSd4dvk2nl5SwP6jZVyUeTpTR2czMiDzEsWC1SmYUPB6zL7b41mtQDgcPFbGn5YW8OdlWzlcWs6Ic7py3+hshvbqFO/QAsuSggk0r8fsuz2e1Qoktj2HS3ny7S3MWbGdkrIKrux7JlNHZ3Nh2mnxDi3wrKPZBNrDDztj9Gtqzph9t8fz+n1NbOw4UMKDL69n+CM5PLWkgCv6nMm//nMEj98+xBKCS3amYALN6zH7bo9ntQKJJW/vUWbk5vHq2l0kiXDD4DTuHdmbjM6JMS9RkFhHszEmYX2ws5jpOXm89eEntE1uxVeHOvMSndUxJd6hBY7VKbRwiTDGPhFiNMG0ausB7vjTSq754xKWbN7PlFFZLH1gDD/7Uh9LCM1kl49CKBHG2CdCjCZYVJUlefuZtjCPFQUH6HRqG75/5bncfmkGHVJaxrxEsWCXj0IoEcbYJ0KMJhgqK5V/b9zDjJw81hUWc2aHtkwakcWtQ3uS2sb+rnXL6hRasEQYY58IMZr4Kq+o5B/rd1fPS5TeKZVfXn8B1w/q0aKnofCbJYUQSoQx9okQo4mP4+UV/P29ncxalM+2ohLOPqMdj908gGsu7EZyknWD+s2SQgglwnz8iRCjia1PyyqYu9KZl+iTw6Vc0KMjs24bzBV9zrR5iWLIkkIIJcIY+0SI0cTG4dITPPuOMy9R0bEyhvbqxKM3Xsjws7u0mHmJgsQ6mo0xcXHgWBlPLyngmXe2cqS0nJHndOW+MdlclGnzEvkh7nUKIpIiIitFZJ2IfCgi/y/KPneIyD4RWRu53eNXPCZ4pkyB5GQQce6nTGnefvGse7CaC/f2HC7lF/M2MOxXC5mem8fl2V2Y983LeeauoZYQAqDBy0ci0ha4Acisub+qPtTAS48DY1T1qIi0BpaIyJuqurzWfi+q6n2NC9skuilTYObMk48rKk4+njGj8fvFs+7Bai7c2XGghJmL8nlpVSEVqlzbvzuTR2Vx9pnt4x2aqaHBy0ci8hZQDKwGKqq2q+pvXL+JSCqwBJisqitqbL8DGNKYpGCXj8IhOdn5BV9bUhKUlzd+v3jWPVjNRf027znCzNx8Xl3nzEt045A07h2RRXrn1IZfbDzjZZ1Cmqpe1cQgknCSSTYwvWZCqOEGERkBbAL+U1V3RDnOJGASQLqNWQyFaL/oo213u1886x6s5iK69YXOvET/3PAJKclJ3HlZJvcMt3mJgs5NUlgmIheo6vrGHlxVK4ABInIa8LKI9FPVD2rs8jowV1WPi8i9wDPAmCjHmQ3MBudMobFxmOBJSqr7DKAp+8Wz7sFqLj5rZcEBpufksWjTPtqnJHPf6GzuHNaLTqe2iXdoxoU6O5pFZL2IvA9cDrwnIh+LyPs1trumqoeAXOCqWtuLVPV45OETwOBGRW8SVtU194a2u90vnusf2NoLzrxEizbt46ZZ73DT4+/wwc5ivn/luSz94Ri+e8W5lhASSH1nCtc058Ai0hU4oaqHROQUYBzwSK19uqnq7sjDCcDG5rynSRxVncSzZztnAklJzi/6mp3HjdkvnnUPLbnmorJS+deGPUzPyWP9zmK6dUzh51/qwy0XpXNKG5uKIhG56Wh+VlVvb2hblNddiHM5KAnnjOT/VPUhEXkIWKWqr4nIL3GSQTlwAKcj+qP6jmsdzcbEX3lFJfPe382M3Dw27TlKRudUJo/M4vpBabRJtqkogsjLjua+tQ6chIvLPKr6PjAwyvaf1fj5R8CPXMRgjAmA4+UV/G21My/R9gMlnHNmO35/ywDGX2DzEoVFfX0KPxKRI8CFInI4cjsC7AVejVmEpkn8KKZyW0Tm9fHctsXrNnvd3oRQMAdeyYTnWzn3Bc6HWFJWzlNLChj5aC4/fnk9p6e2Zvbtg3nr/hFcO6CHJYQwUdV6b8AvG9onlrfBgwerqd9zz6mmpqrCyVtqqrO9qSZP/uzxqm6TJ/t7PLdt8brNXrc3IWx5TvWFVNU5VN8qX0jVt17/tQ586F+a8cA8vWnWMl28aa9WVlbGO1rTSDiX7Rv8HVtnn4KIDGogmbznfYpqmPUpNMyPYiq3RWReH89tW7xus9ftTQivZELJ5z/EwrKu/FTnMXV0NkNsGoqE5bZPob6kkBP5MQUYAqwDBLgQWKGql3sUa6NYUmhYq1bO37W1iUBlZdOOWd9klU2ZU9Ht8dy2xes2e93ehPB8K+DzjVME+WoT/+GYwGj2hHiqOlpVRwPbgEGqOkRVB+N0Hud5F6rxWl1FU80ppqpdLNbQdq+O57YtXrfZ6/YG3baiYxzkrKjPSWoLrcJrodz0Dp2nNaqZ1alIHuBfSKa5/CimcltE5vXx3LbF6zZ73d6g2rTnCN9+YQ2jf53LLwpvo4xaU1AkpUL/FlSFZ1x1NM8FngRGASNxKo/nuumw8ONmHc3uPPecakaGqohz35xO5iqTJ6smJTkdrklJze90dXs8t23xus1etzdI3t9xSCf95V3NeGCenv/TN/W/532oe4o/dTqbX85QnSPO/RYP/uGYQKC5Hc1VRCQFmAyMiGxaDMxU1VJ/0lT9rE/BmKZbsaWI6bn5LN60jw4pydwxrBd3XpbJ6TYNReh5tsiOqpaq6u9U9cuR2+/ilRBMuMSr/qClUVVyP97LV2Yt4+bZy9mwq5gHrjqPpT8cw3e+cE5wEkIdNRImtuqsaBaR/1PVm0RkPVGGJKjqhb5GZkLN7cI0toBN01VWKv/88BOm5+bxwc7DdO+Ywn99qQ83B3FeooI5sHISVES+6JJtzmOAXvZFx1J9Q1K7qepuEcmI9ryqRhkV7j+7fBQO8ao/aAnKKyp5bd0uZuTmk7f3KL26nMrkkVlcN7BHcOclqqNGgtQMuG5rrKMJpWbPfaQnZy8dC7ytqpu9Cs4YtwvT2AI27h0vr+Cl1YXMWpTPjgOfct5Z7fnDrQMZf0E3klrVU3gRBCV1fKF1bTe+cTMhXiZwW+SMYTXwNk6SWOtnYCbc3C5MYwvYNKykrJznV2znibe3sOfwcfr3PI2fXdOXseedQaugJ4Mqqel1nCnYFx1rbjqaf6aqY4B+OOssfx8nORjTZPGqPwiT4k9P8McFmxn2q4X89z820rtLO+bcczGvTLmML/Q5M3ESAji1EEm1vmirkYiLBs8UROQnwDCgHbAG+B7O2YIxTeZ2YZqWvIBNXfYfPc5TSwp49p1tHD1ezpjzzmDq6GwGZ5we79Carqozed2DziWj1HQnIVgnc8y5qVN4D2cRnH8Ai4Dl8RySah3NpqXaXfwpjy/awgvvbud4eSVXX9CNKaOy6Nu9Y7xDMwnAyzqFQTidzSuBLwDrRWRJ80M0NXk9Ft/t8eK5ZoDVH7izdf8xfvi39xnxaA7PLd/GNRd2Z/53RjL9q4PcJ4Qw1QCEqS1uxbDNbi4f9QOG40xxMQTYgV0+8pTXY/HdHm/KFJg58+TjioqTj2uvgew1qz9o2MefHGFGbh6vr9tFclIrbh2azqQRvUk7PbXhF9cUphqAMLXFrRi32c3lo6rLRkuAd1X1hOdRNEIYLx95PRbf7fHiuWaA1R/Ubd2OQ0zLyePfG/Zwapskbrskg7uH9+KM9ikNvziaMNUAhKktbnnUZs/WaFbV8a7f1TSJ12Px3R4vWkKob7uXrP7gs1SVFQUHmJ6Tx9ub99PxlNbcP/Zs7hyWyWmpzZyGIkw1AGFqi1sxbrObOgXjM6/H4rs9XlJS3WcKfrP6A4czL9E+pufksWrbQbq0a8uPvngeEy/JoF1bj/57hqkGIExtcSvGbQ5ozXvL4vVYfLfHi+eaAS29/qCyUnlj/W6u+eMS7vzzu+wuLuWha/uy5IHRfGNklncJAcJVAxCmtrgV6za7mV87SLewrqfg9VoAbo8XzzUD/FjzIejKyiv0pVU7dMyvczTjgXk6+n9z9MV3t+vxExX+vnGY1kkIU1vc8qDNNHc9BRF5nWgLtp5MJhP8SVP1C2NHswm/0hMV/HV1IY8vyqfwoDMv0X1jsvlivwSYl8iEghd1Cr8GflPPzQSYHzUAbo85bpxT91B1Gzeu+e+dqI4dL+eJxVsY8WgOP33lA7q2b8tTXxvCm/cP55oLu1tCSCQrp8DcZHhenPuVHhT1BLDmor5ZUhfFMhDjHT9qANwec9w4WLDgs69dsMDZPn9+0947ERWXnODPy7byp2UFHCo5wbDszjx28wAuzeqMiCWChLNyCuTVKOrRipOPhzaxqCegNRdu6hTOBn4J9IGTq3qram9/Q4vOLh81zI8aALfHrO/3XQP/1EJh3xFnXqLnljvzEo07/wymjM5mUHoCz0tknDMDjTJUT5Lg1iYW9cS45sKzOgXgT8DPgd8Bo4E7AftTJ8D8qAGwuoL67Tr0KbMXb2Huyu2UVVQy/oJuTB2dzfndOsQ7NOOFaAmhvu1uBLTmwk1SOEVVF4iIqLPa2n+JyNs4icIEkB81AFZXEF3B/mPMzM3j5TU7UYUvD+zB5FFZ9O7aLt6hGS9JUt1nCk0V0JoLN3UKpSLSCtgsIveJyJeBM3yOyzSDHzUAbo85dmz019e1PVF99Mlhvjl3DWN/k8ura3fx1aHpLPrBaP73K/0tIYRRVh3FO3VtdyOgNRduzhS+DaQC3wJ+AYwBvuZnUKZ5/FiDwO0x58//fGfz2LHh6WRes/0g03Pymb/RmZdo0ogs7r68F13bt413aMZPVZ3J+bOdMwZJchJCUzuZIbBrSDTY0Vy9o0gHQFX1iL8h1c86mk2sqSrvbCliek4eS/OKOC21NXde1os7LsukY2rreIdnjCuedTSLyBCczub2kcfFwF2qWu+SnCKSAiwG2kbe5yVV/XmtfdoCfwEGA0XAzaq6taGYjIkFVSXn471MW5jHe9sP0bV9W3589Xl89WIP5yUyJmDc9Ck8DUxR1UxVzQSm4iSJhhwHxqhqf2AAcJWIXFJrn7uBg6qajTO66RHXkScItwVfibDgjNsFeRK9zRWVyrz3d/Ho4z/nnOUX8FLH4Xww8BssvXk3k0Y0c14itwVQXhc1Bf148eS2LWFqcz3c/Os+oqrVi+qo6hIRafASUmSujaORh60jt9rXqq4F/ivy80vAtMgop1CMaHdb8JUIC864XZAnkdt8oqKSV9bsZOaifPqdmMejadNIaXUcgHYVO2H1vZDUqunXfN0WQHld1BT048WT27aEqc0NcFO89jucjua5OL/UbwYOAn8DUNX36nltErAayAamq+oDtZ7/ALhKVQsjj/OBi1V1f13HTKQ+BbcFX4mw4IzbBXkSsc2lJyr466odzFq0hZ2HPuX8bh34W/dbST2x8/M7N6ewyG0BlNdFTUE/Xjy5bUsI2uxl8dqAyH3tuoTLcJLEmLpeqKoVwAAROQ14WUT6qeoHNeOM9rLaG0RkEjAJID2BBsa7LfhKhMIwtwvyJFKbjx4vZ87ybTzxdgH7jx5nUPpp/OK6vow+9wxk7q7oL2pOYZHbAiivi5qCfrx4ctuWMLW5AW5WXhvd3DdR1UMikgtcBdRMCoVAT6BQRJKBjsCBKK+fDcwG50yhufHEituCr0QoDHO7IE8itPlQSZkzL9HSrRR/eoLLs7swdfRALund6eS8RH4UFrktgPL6vYN+vHhy25YwtbkBDXY0i8iZIvKUiLwZedxHRO528bqukTMEROQUYBzwUa3dXuNkzcONwMKw9CeA+4KvRFhwxu2CPEFu874jx/nlmxsZ9quFPDZ/MxdlduKVqcN47p6LPz9RnR+FRW4LoLx+76AfL57ctiVMbW5IQwsuAG8CNwHrIo+TgfUuXnchsAZ4H+fs4GeR7Q8BEyI/pwB/BfKAlUDvho6baIvsuF1IJhEWnHG7IE/Q2lx4sER/+sp6PefBN7TXD+fpN59/TzfuLm74hX4s5rJisurzSapzcO5X1PEhev3eQT9ePLltS4K3meYuslNFRN5V1YtEZI2qDoxsW6uqA+p9oU8SqaPZxNeWfUeZmZvPy2t2IgLXD0zj3lFZ9OpyarxDMybmvFhkp8oxEelMpAM4UmtQ3Mz4TC1BHbOfiDbsOszU599j7G8X8dq6Xdx2SQaLvj+aR268MP4JIehj4v2IL+g1Ei2k/sAtN6OPvoNz7T9LRJYCXXGu/xuPBHHMfiJ6b/tBpi/MY8FHe2nXNpl7RzrzEnVpF5B5iYI+Jt6P+IJeI9GC6g/ccjX3UWRk0Lk4Q0g/VtUTfgdWlzBePgrSmP1Eo6q8k1/EtJw8luU78xLdNawXX7s0gPMSBX1MvB/xBb1GIgT1B255OffRV4C3VPVDEfkJMEhE/lvrKVozjROEMfuJRlVZsHEv03PzWLP9EGe0b8tPxp/PrUPTOTWo8xIFfUy8H/EFvUaiBdUfuOWmT+GnqnpERC4HrgSeAWY28BrTCHWNzQ9SnUJQVFQqr6/bxRd//zb3/GUV+44c57+v68fiH4zmnuG9g5sQoO4x7dHGxDfm9V7xIz6v2xL044WAm6RQVW0zHpipqq8CbfwLqeVJhDqFeCsrr+T/3t3BuN8u4ptz13CiopLffKU/Od8bxW2XZJDSuhkrYMVK0MfE+xFf0GskWlL9gUtu/qzaKSKP4xSfPRKZ7tpNMjEu+bEoTliUnqjgxXd3MHuxMy9R3+4dmDlxEFf2PYtWrRJsqXC3i6rEa/EVP+Lzui1BP14IuKlTSMWZnmK9qm4WkW7ABar6r1gEWFsYO5rN5x09Xs5zy7fxZGReoiEZpzN1TDajzun62cpjY4wrnnU0q2oJ8Pcaj3cDu5sXnjHRHTzmzEv052XOvETDz3bmJbq4V6eWlQxWTvF26UevBT0+cIabxuMMIF7v65EA98qZlmTv4VKeXFLAc8u3UVJWwRV9zmTq6Gz69zwt3qHFntt1F+Il6PFB8Gs9Asz1Gs1BYZePwqXwYAmPL9rCi6t2UF5RyZf6d2fKqGzOPat9vEOLH7frLsRL0OOD4Nd6xIGX6ykY47n8fUeZkZPPq2udeYluHJzGN0ZkkRnvaSiCwO26C/ES9Pgg+LUeAWZJwcTUh7uKmZGTzxsf7KZtcituvzSDSSN6063jKfEOLTjcrrsQL0GPD+K3/kEI1l2woaUmJlZvO8hdf36X8X9YwuJN+5g8MoslD4zh51/qawmhNrfrLsRL0OOD4Nd6BJidKRjfqCpL84qYlrOZ5VsOcHpqa753xTncfmkmHU8J2LxEQVLVWRvU0T1Bjw+CX+sRYNbRbDxXWaks+Ggv03LyWLfjEGd2aMvXh/fmqxenk9rG/g4xJh6so9nEXEWlMu/9XczMzeejT47Qs9MpPPzlftw4OI22yTG43pwI48O9jtHreoFE+AyNrywpmGYrK6/k5TWFzMx7pxsCAAARvklEQVTNZ2tRCdlntON3N/fnSxd2JzkpRt1WiTA+3OsYva4XSITP0PjOLh+ZJis9UcELK7cze/EWdhWX0q9HB+4bnc0VfeIwL1GAx4dX8zpGr+sFEuEzNE1ml4+Mb46UnuDZ5dt4ekkB+4+WcVHm6fzP9RcwMp7zEiXC+HCvY/S6XiARPkPjO0sKxrWDx8r409IC/rxsK4dLyxlxTlfuG53N0F6d4h1aYowP9zpGr+sFEuEzNL6zOgXToD2HS3n4HxsY9shC/rAwj0uzOvPafcP4y11Dg5EQIDHGh3sdo9f1AonwGRrf2ZmCqdOOAyXMWpTPX1cVUqHKhP7dmTwqi3PODOC8RIkwPtzrGL2uF0iEz9D4zjqazefk7T3KjNw8Xl27iyQRbhicxuSRWaR3Tm34xcaYQLKOZtNoH+wsZnpOHm99+Altk1vxtUszmTSiN2d1TIl3aMHn9fh+t8ezugLjMUsKhlVbDzAtJ4/cj/fRvm0yU0dlc+ewTDq3axvv0BKD1+P73R7P6gqMD+zyUQulqizJ28+0hXmsKDhAp1PbcPflvbj90gw6pNi8RI3i9fh+t8ezugLTCHb5yERVWan8e+MeZuTksa6wmLM6pPDTa/pw69CeNi9RU3k9vt/t8ayuwPjAfgu0EOUVlfxj/W6m5+Sxac9R0jul8svrL+D6QT1iMy9RmHk9vt/t8ayuwPjA6hRC7nh5BXNXbmfsbxdx/wtrUYXHbh7Awu+O5Nah6ZYQvOD1+H63x7O6AuMDO1MIqU/LnGQwe/EWPjlcygU9OjLrtsFc0efM2M9LFHZej+93ezyrKzA+sI7mkDlceoJn33HmJSo6VsbQXp24b3Q2w8/uEr95iYwxcRf3jmYR6Qn8BTgLqARmq+rva+0zCngVKIhs+ruqPuRXTGF24FgZTy8p4Jl3tnKktJyR53TlvjHZXJQZkGkojDEJwc/LR+XAd1X1PRFpD6wWkX+r6oZa+72tqtf4GEeo7TlcyuzFW3h+xXZKyyu4qu9ZTB2dTb8eHeMdmncSoUDLis2azz6bQPAtKajqbmB35OcjIrIR6AHUTgqmCbYXlTBrcT4vReYlujYyL9HZQZyXqDkSoUDLis2azz6bwIhJn4KIZAKLgX6qerjG9lHA34BCYBfwPVX9sL5jtfQ+hc17jjAjN5/X1jnzEt04JI17R4R4XqJEKNCyYrPms8/Gd3HvU6gRSDucX/zfrpkQIt4DMlT1qIhcDbwCnB3lGJOASQDp6S1zDPb6Qmdeon9u+ISU5CTuvCyTe4a3gHmJEqFAy4rNms8+m8DwNSmISGuchDBHVf9e+/maSUJV3xCRGSLSRVX319pvNjAbnDMFP2MOmpUFzrxEizfto31KMveNzubOYb3odGqbeIcWG4lQoGXFZs1nn01g+Fa8Js74x6eAjar62zr2OSuyHyIyNBJPkV8xJQpVZdGmfdw06x1uevwdPtxZzPevPJelPxzDd684t+UkBEiMAi0rNms++2wCw88zhWHA7cB6EVkb2fZjIB1AVWcBNwKTRaQc+BS4RROtcMJDlZXKvzbsYXpOHut3FtOtYwo//1IfbrkonVPatNDK40Qo0LJis+azzyYwrHgtAMorKnn9/V3MyMln896jZHROZfLILK4flEabZJuJxBjTfIHpaDZ1O15ewd9W72TWony2HyjhnDPb8ftbBjD+gm4kJyVwMmiJ481XTvFuWUxj4siSQhyUlJXz/IrtPPH2FvYcPk7/tI78ZPxgxp0fgnmJWuJ485VTIG/mycdacfKxJQaTYOzyUQwVf3qCZ9/ZytNLt3LgWBkX9+rEfWOyuTw7RPMStcTx5nOTnURQmyTBreWxj8eYKOzyUYAUHT3O00sL+MuybRw5Xs6oc7ty3+hshoRxXqKWON48WkKob7sxAWZJwUe7iz9l9uItzF25nePllXyx31lMGRWyeYlqa4njzSWp7jMFYxKMJQUfbCs6xqxF+by0upBKhesG9GDyqN5knxGyeYmi6f/wZ/sUIPzjzbMmfbZPoeZ2YxKMJQUPbdpzhBk5eby2bhfJSa24+aKefGNEFj07hXReomha4njzqs5kG31kQsA6mj3wfuEhpi3M418b9pDaJomJF6fz9eG9OaNDyOclMsYkDLcdzQk8GD7+Vmwp4vanVjBh2lKWbyniW2OyWfrAGB4c38e3hDBnDmRmQqtWzv2cOb68TWwUzHFGKz3fyrkvSODGhKkt8WKfYSDY5aNGUlVyN+1jRk4e7249SJd2bXjgqvO47ZJ02qe09vW958yBSZOgJHK5fts25zHAxES7OhOmeoYwtSVe7DMMDLt85FJlpfLPDz9hem4eH+w8TLeOKXxjRG9ujuG8RJmZTiKoLSMDtm6NSQjeCVM9Q5jaEi/2GfrO6hQ8cqKiktfX7WJGbj55e4+S2TmVR264gC8PjP28RNvrGOpf1/ZAC1M9Q5jaEi/2GQaGJYU6lJ6o4KXVhcxalE/hwU8576z2/OHWgYy/oBtJcZqKIj09+plCQq47FKZ6hjC1JV7sMwwM62iupaSsnCff3sKIR3P4ySsf0LldW574jyG88a3hTOjfPW4JAeDhhyG11ujW1FRne8IJ0/z5YWpLvNhnGBh2phBR/OkJnlm2lT8tLeBgyQku7d2Z3908gMuyOgdmXqKqzuQHH3QuGaWnOwkh4TqZIVz1DGFqS7zYZxgYLb6jef/R4zy1pIBn39nG0ePljDnvDKaOzmZwxumevYcxxsSbdTQ3YNchZ16iF9515iW6ul83pozOom/3EM9LZExD/FgLoyWur5HAWlxS2Lr/GDNz8/n7mkJU4bqBPZg8Kousru3iHZox8eVHrYDVHyScFpMU8vcd5ffzNzPvfWdeolsuSmfSiN4ta14iY+qz7sHPTmQIzuN1Dzb9F7gfxzS+ajFJYfuBEhZs3MPXh/fm7st72bxExtTmR62A1R8knBaTFEad05VlPxxLx1R/p6IwJmH5UStg9QcJp8XUKYiIJQRj6uNHrYDVHyScFpMUjDEN6DURhs525htCnPuhs5t37d+PYxpftfg6BWOMaQlsPQVjjDGNZknBGGNMNUsKxhhjqllSMMYYU82SgjHGmGqWFIwxxlSzpGCMMaaaJQVjjDHVfEsKItJTRHJEZKOIfCgi90fZR0TkDyKSJyLvi8ggv+IxxhjTMD/PFMqB76rq+cAlwFQR6VNrny8CZ0duk4CZPsZjgqZgDrySCc+3cu4L5sQ7ImNaPN+SgqruVtX3Ij8fATYCPWrtdi3wF3UsB04TkW5+xWQCpGrxlZJtgJ5cfMUSgzFxFZM+BRHJBAYCK2o91QPYUeNxIZ9PHCaM6lt8xRgTN74nBRFpB/wN+LaqHq79dJSXfG6GPhGZJCKrRGTVvn37/AjTxJotvmJMIPmaFESkNU5CmKOqf4+ySyHQs8bjNGBX7Z1UdbaqDlHVIV27dvUnWBNbdS2yYouvGBNXfo4+EuApYKOq/raO3V4D/iMyCukSoFhVd/sVkwkQW3zFmEDycznOYcDtwHoRWRvZ9mMgHUBVZwFvAFcDeUAJcKeP8ZggqVpkZd2DziWj1HQnIdjiK8bElW9JQVWXEL3PoOY+Ckz1KwYTcL0mWhIwJmCsotkYY0w1SwrGGGOqWVIwxhhTzZKCMcaYapYUjDHGVBNnAFDiEJF9wLYmvrwLsN/DcOLJ2hJMYWlLWNoB1pYqGaraYPVvwiWF5hCRVao6JN5xeMHaEkxhaUtY2gHWlsayy0fGGGOqWVIwxhhTraUlhdnxDsBD1pZgCktbwtIOsLY0SovqUzDGGFO/lnamYIwxph6hTQoikiQia0RkXpTn2orIiyKSJyIrIivDBVYDbblDRPaJyNrI7Z54xOiGiGwVkfWROFdFeV5E5A+R7+V9ERkUjzgb4qIdo0SkuMZ38rN4xOmGiJwmIi+JyEcislFELq31fEJ8J+CqLQnxvYjIuTViXCsih0Xk27X28e178XPq7Hi7H2dd6A5RnrsbOKiq2SJyC/AIcHMsg2uk+toC8KKq3hfDeJpjtKrWNc76i8DZkdvFwMzIfRDV1w6At1X1mphF03S/B95S1RtFpA1Qa5GLhPpOGmoLJMD3oqofAwPA+YMQ2Am8XGs3376XUJ4piEgaMB54so5drgWeifz8EjA2sihQ4LhoS5hcC/xFHcuB00SkW7yDCisR6QCMwFkMC1UtU9VDtXZLiO/EZVsS0VggX1VrF+z69r2EMikAjwE/ACrreL4HsANAVcuBYqBzbEJrtIbaAnBD5BTyJRHpWc9+8abAv0RktYhMivJ89fcSURjZFjQNtQPgUhFZJyJvikjfWAbXCL2BfcCfIpcnnxSRU2vtkyjfiZu2QGJ8LzXdAsyNst237yV0SUFErgH2qurq+naLsi1ww7BctuV1IFNVLwTmc/IMKIiGqeognFPfqSIyotbzCfG90HA73sOZUqA/8EfglVgH6FIyMAiYqaoDgWPAD2vtkyjfiZu2JMr3AkDkEtgE4K/Rno6yzZPvJXRJAWcZ0AkishV4ARgjIs/V2qcQ6AkgIslAR+BALIN0qcG2qGqRqh6PPHwCGBzbEN1T1V2R+70410iH1tql+nuJSAN2xSY69xpqh6oeVtWjkZ/fAFqLSJeYB9qwQqBQVVdEHr+E84u19j6B/05w0ZYE+l6qfBF4T1X3RHnOt+8ldElBVX+kqmmqmolz6rVQVW+rtdtrwNciP98Y2Sdwf/24aUut64gTcDqkA0dEThWR9lU/A1cAH9Ta7TXgPyIjKy4BilV1d4xDrZebdojIWVV9VCIyFOf/WVGsY22Iqn4C7BCRcyObxgIbau0W+O8E3LUlUb6XGm4l+qUj8PF7CfPoo88QkYeAVar6Gk5n1LMikodzhnBLXINrpFpt+ZaITADKcdpyRzxjq8eZwMuR/5PJwPOq+paI3AugqrOAN4CrgTygBLgzTrHWx007bgQmi0g58ClwSxD/6Ij4JjAncqliC3BnAn4nVRpqS8J8LyKSCnwB+EaNbTH5Xqyi2RhjTLXQXT4yxhjTdJYUjDHGVLOkYIwxppolBWOMMdUsKRhjjKlmScGYRorMthltxtqo2z14v+tEpE+Nx7kiEoo1h03wWFIwJviuA/o0uJcxHrCkYEInUnX8j8jEZx+IyM2R7YNFZFFkIrt/VlWDR/7yfkxElkX2HxrZPjSybU3k/tz63jdKDE+LyLuR118b2X6HiPxdRN4Skc0i8miN19wtIpsi8TwhItNE5DKcSvX/FWdu/azI7l8RkZWR/Yd79NEZ03Iqmk2LchWwS1XHA4hIRxFpjTMJ2rWqui+SKB4G7oq85lRVvSwyud3TQD/gI2CEqpaLyDjgf4AbXMbwIM60JHeJyGnAShGZH3luADAQOA58LCJ/BCqAn+LM13MEWAisU9VlIvIaME9VX4q0ByBZVYeKyNXAz4FxTfmgjKnNkoIJo/XAr0XkEZxfpm+LSD+cX/T/jvxSTQJqzhUzF0BVF4tIh8gv8vbAMyJyNs4MlK0bEcMVOJMZfi/yOAVIj/y8QFWLAURkA5ABdAEWqeqByPa/AufUc/y/R+5XA5mNiMuYellSMKGjqptEZDDO3DC/FJF/4cxm+qGqXlrXy6I8/gWQo6pfFmfJ1txGhCHADZFVtE5uFLkY5wyhSgXO/8PGLvJUdYyq1xvjCetTMKEjIt2BElV9Dvg1ziWZj4GuElm3V0Ray2cXWanqd7gcZ8bJYpwp1XdGnr+jkWH8E/hmjVk5Bzaw/0pgpIicLs507jUvUx3BOWsxxnf2F4YJowtwOmYrgRPAZFUtE5EbgT+ISEecf/uPAR9GXnNQRJbhrINd1c/wKM7lo+/gXONvjF9Ejv9+JDFsBepcG1hVd4rI/wArcObF34CzIiA4a2k8ISLfwpnp0xjf2CyppsUTkVzge6q6Ks5xtFPVo5EzhZeBp1W19oLtxvjKLh8ZExz/JSJrcRbtKSDgy0WacLIzBWOMMdXsTMEYY0w1SwrGGGOqWVIwxhhTzZKCMcaYapYUjDHGVLOkYIwxptr/B/cTZEbiZPByAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MNz9NEgH5OXi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 143
        },
        "outputId": "d44ffab4-3864-44c8-9448-26a09ad5c8ff"
      },
      "source": [
        "My_skl = LogisticRegression(max_iter=200)\n",
        "My_skl.fit(My_X, My_y)"
      ],
      "execution_count": 113,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
            "  FutureWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "                   intercept_scaling=1, l1_ratio=None, max_iter=200,\n",
              "                   multi_class='warn', n_jobs=None, penalty='l2',\n",
              "                   random_state=None, solver='warn', tol=0.0001, verbose=0,\n",
              "                   warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 113
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ScD2LyfB5cBa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "4b82f13c-0da1-4c48-d00e-929284bc2478"
      },
      "source": [
        "My_skl.predict(np.array([[1,2,-2]]))"
      ],
      "execution_count": 115,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 115
        }
      ]
    }
  ]
}